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INPUT DEVICE, INFORMATION DEVICE, AND CONTROL INFORMATION 

GENERATION METHOD 



Japanese Patent Application No. 2002-277455, filed on September 24, 2002, is 
5 hereby incorporated by reference in its entirety. 

BACKGROUND OF THE INVENTION 
The present invention relates to an input device, an information device 
including the input device, and a control information generation method. 
10 An input device is used as an operating section of an electronic instrument 

(information instrument or information device). For example, if a user operates the 
input device, a pointer displayed in a display section is moved or an image of the 
display section is scrolled in the electronic instrument by using control information 
(operation information) output from the input device. It is necessary that the input 
15 device not decrease operability of the user. 

BRIEF SUMMARY OF THE INVENTION 
According to a first aspect of the present invention, there is provided an input 
device which generates control information by moving an object to be detected, 
20 comprising: 

an image capture section which captures an image of the object to be detected; 

a feature point extraction section which extracts a feature point of the image 
captured by the image capture section; 

a difference calculation section which calculates a difference between a 
25 reference position and a position of the feature point; and 

a control information output section which outputs the control information 
corresponding to the calculated difference. 
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According to a second aspect of the present invention, there is provided an 
input device which generates control information by moving an object to be detected, 
comprising: 

an image capture section which captures an image of the object to be detected; 
5 an area calculation section which calculates an area of the image captured by 

the image capture section; 

a difference calculation section which calculates a difference between an area 
of the image of the object to be detected in a frame at least one frame before the current 
frame and the area calculated by the area calculation section; and 
10 a control information output section which outputs the control information 

corresponding to the calculated difference. 

According to a third aspect of the present invention, there is provided an input 
device which generates control information by moving an object to be detected, 
comprising: 

15 an image capture section which includes a detection surface and captures an 

image of the object to be detected which is in contact with the detection surface; 

a center-of-gravity calculation section which calculates a center of gravity of 
the image captured by the image capture section; 

a feature point extraction section which extracts a feature point of the captured 

20 image; 

a center-of-gravity movement detection section which calculates a first 
difference between a center of gravity of an image of the object to be detected in a 
frame at least one frame before the current frame and the center of gravity calculated by 
the center-of-gravity calculation section; 
25 a feature point position movement detection section which calculates a second 

difference between a position of a feature point of an image of the object to be detected 
in a frame at least one frame before the current frame and a position of the feature point 
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extracted by the feature point extraction section; and 

a rotation angle detection section which detects a rotation angle around one of 
a first axis and a second axis which are perpendicular to each other on the detection 
surface based on the first difference and the second difference, 
5 wherein the rotation angle detection section calculates the rotation angle 

around the second axis by subtracting a first axis direction component of the second 
difference from a first axis direction component of the first difference, calculates the 
rotation angle around the first axis by subtracting a second axis direction component of 
the second difference from a second axis direction component of the first difference, 

10 and outputs the control information corresponding to the rotation angle around the first 
axis or the second axis. 

According to a fourth aspect of the present invention, there is provided An 
information device comprising: any of the above input devices; and a processing section 
which performs control processing based on the control information from the input 

15 device. 

According to a fifth aspect of the present invention, there is provided a control 
information generation method of generating control information by using a captured 
image of an object to be detected, the control information generation method 
comprising: 

20 extracting a feature point of an image of the object to be detected; 

calculating a difference between a reference position and a position of the 
feature point; and 

outputting the control information corresponding to the difference. 
According to a sixth aspect of the present invention, there is provided a control 
25 information generation method of generating control information by using a captured 
image of an object to be detected, the control information generation method 
comprising: 
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calculating an area of the image of the object to be detected which is in contact 
with a detection surface; 

calculating a difference between the calculated area and an area of an image of 
the object to be detected in a frame at least one frame before the current frame; and 
5 outputting the control information corresponding to the difference in response 

to the movement of the object to be detected in a direction perpendicular to the 
detection surface. 

According to a seventh aspect of the present invention, there is provided a 
control information generation method of generating control information by using a 
10 captured image of an object to be detected, the control information generation method 
comprising: 

calculating a center of gravity of the image of the object to be detected which is 
in contact with a detection surface and extracting a feature point of the image; 

calculating a difference between the center of gravity and a center of gravity of 
15 an image of the object to be detected in a frame at least one frame before the current 
frame and calculating a difference between a position of the feature point and a position 
of a feature point of an image of the object to be detected in the frame at least one frame 
before the current frame; 

detecting a rotation angle around one of a first axis and second axis which are 
20 perpendicular to each other on the detection surface, based on the difference in the 
center of gravity and the difference in the feature point; and 

outputting the control information corresponding to the rotation angle. 



BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWING 
25 FIG. 1 is a block diagram showing configuration of the input device according 

to one embodiment of the present invention. 

FIG 2 is a diagram for illustrating control information of six-axis directions. 



4 



EP-64380US 

FIG. 3 is a diagram schematically showing the input device using a fingerprint 

sensor. 

FIG. 4 is a block diagram showing hardware configuration of the input device. 

FIG. 5 is a circuit diagram showing the fingerprint sensor. 
5 FIG. 6 is a cross-sectional view showing the capacitance detection element. 

FIG. 7 is an equivalent circuit diagram of the capacitance detection element 
when a ridge of a fingerprint touches the capacitance detection element. 

FIG. 8 is an equivalent circuit diagram of the capacitance detection element 
when a valley of a fingerprint faces the capacitance detection element. 
10 FIGS. 9A and 9B are diagrams showing feature points of a fingerprint. 

FIG. 10 is a flowchart showing a processing flow of the input device. 

FIGS. 11 A, 11B, and 11C are diagrams showing changes in a fingerprint image 
when a finger is moved in the Z axis direction. 

FIG 12 is a diagram for illustrating a method of reducing the processing load 
15 of calculating an area of a fingerprint image. 

FIGS. 13A and 13B are diagrams showing comparing processing of feature 
points of a fingerprint. 

FIGS. 14A ? 14B, and 14C are diagrams for illustrating changes in a fingerprint 
image when a finger is moved around the Z axis. 
20 FIGS. 15 A, 15B, and 15C are diagrams illustrating changes in a fingerprint 

image when a finger is moved around the Y axis. 

FIGS. 16A 5 16B, and 16C are diagrams illustrating changes in a fingerprint 
image when a finger is moved around the X axis. 

FIGS. 17A 5 17B, and 17C are diagrams illustrating changes in a fingerprint 
25 image when a finger is moved in the X axis direction. 

FIGS. 18A, 18B, and 18C are diagrams illustrating changes in a fingerprint 
image when a finger is moved in the Y axis direction. 
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FIG. 19 is a block diagram showing configuration of an IC card. 

DETAILED DESCRIPTION OF THE EMBODIMENT 
Embodiments of the present invention are described below. Note that the 
5 embodiments described hereunder do not in any way limit the scope of the invention 
defined by the claims laid out herein. Note also that all of the elements described 
below should not be taken as essential requirements for the present invention. 

An input device provided with improved operability when indicating an 
arbitrary position in a three-dimensional space has been proposed. In this input device, 
10 a reference point is set. In the case where the indicated position is not displayed on the 
screen, the viewpoint is moved by combination of movement around the reference point 
and movement along a straight line which connects the reference point with the 
viewpoint, and the three-dimensional space is regenerated (displayed) from the 
viewpoint after the movement. In the case where the indicated position appears on the 
15 screen, a cursor is moved on the screen (Japanese Patent Application Laid-open No. 
5-40571, for example). The above operation makes it unnecessary to perform the 
operation in the six-axis directions in this input device. 

However, it is difficult to apply the input device disclosed in Japanese Patent 
Application Laid-open No. 5-40571 to a portable information instrument. In the case 
20 of applying an input device to a portable information instrument, the input device must 
have a configuration which allows operation by use of a battery and reduction of the 
size. It is desirable that the input device having the above-described function be 
applied not only to a three-dimensional CAD device or a virtual reality experience 
device which performs advanced information processing, but also to a portable 
25 telephone or a PDA while reducing the processing load. 

According to the following embodiments, an input device capable of further 
improving operability without increasing the processing load, an information device, 
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and a control information generation method can be provided. 

According to the following embodiments, an input device which is extremely 
small and lightweight and is capable of further improving operability, an information 
device, and a control information generation method can be provided. 
5 The embodiments of the present invention are described below in detail with 

reference to the drawings. 

1 . Input Device 

FIG. 1 shows configuration of the input device according to one embodiment of 
10 the present invention. An input device 10 in this embodiment is capable of outputting 

control information (operation information) in the six-axis directions based on the 

change in a captured image. Therefore, the input device 1 0 includes an image capture 

section 20, an image analysis section 30, a difference calculation section 40, and a 

control information output section 50. 
15 The image capture section 20 captures a two-dimensional or three-dimensional 

object to be detected which is moved by a user as an image (two-dimensional 

information) through a detection surface (sensor surface), and generates image 

information in each frame. 

The image analysis section 30 analyzes the image captured by the image 
20 capture section 20, and outputs the analysis result to the difference calculation section 

40. In more detail, the image analysis section 30 includes a feature point extraction 

section 32, a center-of-gravity calculation section 34, and an area calculation section 36. 

The feature point extraction section 32 extracts a feature point of the image 

captured by the image capture section 20. The feature point used herein refers to a 
25 position (region) characteristic of the image which can be referred to for specifying the 

moving distance, moving direction, or rotation angle by comparing the images before 

and after movement. 
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The center-of-gravity calculation section 34 calculates the center of gravity of 
the image captured by the image capture section 20 or a position equivalent to the center 
of gravity. In more detail, the center-of-gravity calculation section 34 calculates the 
center of the image of an object to be detected or a position equivalent to the center. 
5 The area calculation section 36 calculates the area of the image captured by the 

image capture section 20 or a value equivalent to the area. In more detail, the area 
calculation section 36 calculates the area of the object to be detected or a value 
equivalent to the area of the object to be detected. 

The difference calculation section 40 refers to an image in a frame at least one 

10 frame before the current frame (given reference position of the image) stored in a 
memory 48, compares the image with the analysis result from the image analysis 
section 30, and outputs the difference to the control information output section 50. In 
more detail, the difference calculation section 40 includes a feature point movement 
detection section 42, a center-of-gravity movement detection section 44, and an area 

15 change detection section 46. 

The feature point movement detection section 42 compares the feature point of 
the image of the object to be detected in the current frame extracted by the feature point 
extraction section 32 with the feature point of the image of the object to be detected in a 
frame at least one frame before the current frame stored in the memory 48, and detects 

20 the change in position as a difference (second difference). After the difference is 
calculated, the feature point of the image of the object to be detected in the current 
frame is stored in the memory 48, and used to detect the change in position of a feature 
point of an image of the object to be detected in the next frame. 

The center-of-gravity movement detection section 44 compares the center of 

25 gravity of the image of the object to be detected in the current frame calculated by the 
center-of-gravity calculation section 34 with the center of gravity of the image of the 
object to be detected in a frame at least one frame before the current frame stored in the 
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memory 48, and detects the change in the center of gravity as a difference (first 
difference). After the difference is calculated, the center of gravity of the image of the 
object to be detected in the current frame is stored in the memory 48, and used to detect 
the change in the center of gravity of an image of the object to be detected in the next 
5 frame. 

The area change detection section 46 compares the area of the image of the 
object to be detected in the current frame calculated by the area calculation section 36 
with the area of an image of the object to be detected in a frame at least one frame 
before the current frame stored in the memory 48, and detects the change in the area as a 

10 difference. After the difference is calculated, the area of the image of the object to be 
detected in the current frame is stored in the memory 48, and used to detect the change 
in the area of an image of the object to be detected in the next frame. 

The control information output section 50 outputs the control information 
corresponding to the difference between the image of the object to be detected in the 

15 current frame and the image of the object to be detected in the frame at least one frame 
before the current frame which is calculated by the difference calculation section 40. 
The control information is control information in the six-axis directions as described 
above. 

FIG. 2 is a diagram for illustrating the control information of the six-axis 
20 directions. The control information in the six-axis directions is information indicated 
for the six-axis directions including changes Ax and Ay in the directions of the X axis 
and the Y axis (first and second axes) which intersect at right angles on the detection 
surface (sensor surface) 22 of the image capture section 20 (or on a plane parallel to the 
detection surface), a change Az in the Z axis (third axis) direction perpendicular to the 
25 detection surface, a rotation angle a around the X axis, a rotation angle p around the Y 
axis, and a rotation angle y around the Z axis. As shown in FIG. 2, a (+) direction and 
a (-) direction are specified for the change Ax in the X axis direction, the change Ay in 
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the Y axis direction, the change Az in the Z axis direction, the rotation angle a around 
the X axis, the rotation angle P around the Y axis, and the rotation angle y around the Z 
axis. 

In this embodiment, the movement of the center of gravity of the image of the 
5 object to be detected and the movement of the feature point of the image of the object to 
be detected are used to calculate the rotation angle around each axis in addition to the 
movement in the X axis, Y axis, and Z axis directions specified on the detection surface 
while reducing the processing load. Therefore, the control information output section 
50 includes an X axis movement detection section 52, a Y axis movement detection 

10 section 54, an X axis rotation detection section 56, a Y axis rotation detection section 58, 
a Z axis rotation detection section 60, and a Z axis movement detection section 62. 
The control information output section 50 outputs the control information 
corresponding to output of each section. 

The X axis movement detection section 52 detects the movement of the object 

15 to be detected on the detection surface in the X axis direction. The X axis movement 
detection section 52 also detects the moving amount Ax of the object to be detected in 
the X axis direction on the detection surface in the case where the object to be detected 
is rotated while being slid on the detection surface, taking rotation of the object to be 
detected around the Y axis into consideration. In more detail, provided that the 

20 moving amount of the feature point of the image of the object to be detected in the X 
axis direction is AX2 (first axis direction component of the second difference) and the 
rotation angle around the Y axis is y (rotation angle around the second axis), the X axis 
movement detection section 52 calculates the moving amount Ax of the object to be 
detected on the detection surface in the X axis direction as expressed by the following 

25 equation (1). 

Ax = AX2-a-y (where 0 ^ a ^ 1) (1) 
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The coefficient a (given first coefficient) is a weighting coefficient of the 
rotation angle y around the Y axis. If the weighting coefficient a is 0 5 the rotation 
angle around the Y axis is not taken into consideration. Therefore, this applies to the 
5 output Ax in the case where the object to be detected is moved on the detection surface 
without being rotated. This also applies to the case where the object to be detected is 
rotated without being slid on the detection surface so that the rotation angle y is output 
and Ax is set to "0". If the weighting coefficient a is 1 , this applies to the case where 
the object to be detected is merely rotated while being slid on the detection surface and 

10 is not moved. It is preferable that the weighting coefficient a be determined by adding 
the user's peculiar operation in the X axis direction, for example. 

The Y axis movement detection section 54 detects the movement of the object 
to be detected on the detection surface in the Y axis direction. The Y axis movement 
detection section 54 also detects the moving amount of the object to be detected on the 

15 detection surface in the Y axis direction in the case where the object to be detected is 
rotated while being slid on the detection surface, taking rotation of the object to be 
detected around the X axis into consideration. In more detail, provided that the 
moving amount of the feature point of the image of the object to be detected in the Y 
axis direction is AY2 (second axis direction component of the second difference) and 

20 the rotation angle around the X axis is a (rotation angle around the first axis), the Y axis 
movement detection section 54 calculates the moving amount Ay of the object to be 
detected on the detection surface in the Y axis direction as expressed by the following 
equation (2). 

25 Ay = AY2-b.a (where 0 ^ b ^ 1) (2) 

The coefficient b (given second coefficient) is a weighting coefficient of the 
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rotation angle a around the X axis. If the weighting coefficient b is 0 5 the rotation 
angle around the X axis is not taken into consideration. Therefore, this applies to the 
output Ay in the case where the object to be detected is moved on the detection surface 
without being rotated. This also applies to the case where the object to be detected is 
5 rotated without being slid on the detection surface so that the rotation angle a is output 
and Ay is set to "0". If the weighting coefficient b is 1 5 this applies to the case where 
the object to be detected is merely rotated while being slid on the detection surface and 
is not moved. It is preferable that the weighting coefficient b be determined by adding 
the user's peculiar operation in the Y axis direction, for example. 

10 The X axis rotation detection section 56 detects the rotation angle a around the 

X axis. In more detail, provided that the moving amount of the center of gravity of the 
image of the object to be detected in the Y axis direction is AY1 (second axis direction 
component of the first difference) on condition that the rotational radius is sufficiently 
large or the rotation angle is sufficiently small, the X axis rotation detection section 56 

15 calculates the rotation angle a around the X axis as expressed by the following equation 
(3). 

a = - (AY1 - AY2) (3) 

20 The negative sign is added to the equation (3) depending on the positive and 

negative directions of the rotation angle specified in FIG. 2. Therefore, if the positive 
and negative directions of the rotation angle a in FIG. 2 are reversed, the sign of the 
equation (3) becomes positive. 

The X axis rotation detection section 56 may calculate the rotation angle a 

25 around the X axis from the equations (2) and (3) based on the moving amount AY2 of 
the feature point of the image of the object to be detected in the Y axis direction 
(difference calculated by the feature point movement detection section 42) and the 
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moving amount AY1 of the center of gravity of the image of the object to be detected in 
the Y axis direction (difference calculated by the center-of-gravity movement detection 
section 44). 

The Y axis rotation detection section 58 detects the rotation angle y around the 
5 Y axis. In more detail, provided that the moving amount of the center of gravity of the 
image of the object to be detected in the X axis direction is AX1 (first axis direction 
component of the first difference) on condition that the rotational radius is sufficiently 
large or the rotation angle is sufficiently small, the Y axis rotation detection section 58 
calculates the rotation angle y around the Y axis as expressed by the following equation 
10 (4). 

y = AX1 - AX2 (4) 

The Z axis rotation detection section 60 detects the rotation angle P around the 
15 Z axis. In more detail, in the case where the feature points of the images of the object 
to be detected before and after movement are associated with each other by rotating the 
feature point around a given reference point, the Z axis rotation detection section 60 
calculates the angle at which the feature point is rotated around the reference point as 
the rotation angle p. 

20 The Y axis rotation detection section 58 may calculate the rotation angle y 

around the Y axis from the equations (1) and (4) based on the moving amount AX2 of 
the feature point of the image of the object to be detected in the X axis direction 
(difference calculated by the feature point movement detection section 42) and the 
moving amount AX1 of the center of gravity of the image of the object to be detected in 

25 the X axis direction (difference calculated by the center-of-gravity movement detection 
section 44). 

The Z axis movement detection section 62 detects the movement of the object 
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to be detected in the direction perpendicular to the detection surface. In more detail, 
the Z axis movement detection section 62 calculates the moving amount Az in the Z axis 
direction by utilizing a phenomenon in which the image of the object to be detected 
changes depending on the pressure applied to the detection surface. Provided that the 
5 area of the image of the object to be detected before movement is SI and the area of the 
image of the object to be detected after movement is S2, the Z axis movement detection 
section 62 calculates the moving amount Az in the Z axis direction as expressed by the 
following equation (5). 

10 Az = c(Sl-S2) (5) 

The coefficient c is a coefficient for associating the change in the area of the 
image of the object to be detected with the moving amount in the Z axis direction. It is 
preferable that the coefficient c be determined by adding the user's peculiar operation in 
15 the Z axis direction, for example. 

The input device is described below in detail. The input device described 
below uses a fingerprint sensor. However, the present invention is not limited thereto. 

FIG 3 schematically shows the input device having a fingerprint sensor. The 
input device is mounted on an IC card (or an information device in a broad sense) 64 is 
20 shown in this figure. The IC card 64 includes a CPU and a memory device. This 
enables the IC card 64 to be provided with improved security protection and to store a 
large amount of advanced information by information processing. Information 
processing in which various types of operation of the user are reflected can be 
performed by using an extremely small and lightweight configuration by using the input 
25 device in this embodiment. 

In FIG 3, a fingerprint image is captured by allowing a finger (object to be 
detected in a broad sense) 66 of the user on which a fingerprint pattern is formed to 
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come in contact with the detection surface 22 of the fingerprint sensor as the input 
device. The control information corresponding to movement of the finger 66 by the 
user in the six-axis directions which is detected in the three-dimensional space specified 
on the detection surface 22 is output. Processing based on the control information is 
5 performed in the IC card 64. In the case where a liquid crystal panel is provided to the 
IC card 64, display control such as movement of a pointer displayed on the liquid 
crystal panel or scrolling of the display image is performed. In the case where the 
input device is applied to a three-dimensional CAD device, rotation of the object of 
operation or movement of the viewpoint is controlled. 

10 FIG. 4 shows hardware configuration of the input device. In an input device 

68, a CPU 72, a ROM 74, a RAM 76, and a fingerprint sensor interface (I/F) circuit 78 
are connected with a bus 70. A fingerprint sensor 80 is connected with the fingerprint 
sensor I/F circuit 78. A USB I/F circuit 82 is connected with the bus 70. The USB 
I/F circuit 82 is connected with a host device or a peripheral device defined in the USB 

15 standard such as a personal computer 84 outside the input device. 

The function of the image capture section 20 shown in FIG. 1 is mainly 
realized by the fingerprint sensor 80 and the fingerprint sensor I/F circuit 78. A 
fingerprint image captured by the fingerprint sensor 80 is stored in the RAM 76 through 
the fingerprint sensor I/F circuit 78. The functions of the image analysis section 30, 

20 the difference calculation section 40, and the control information output section 50 
shown in FIG. 1 are realized by the CPU 72 and a software program stored in the ROM 
74 or RAM 76. The function of the memory 48 shown in FIG 1 is realized by the 
RAM 76. 

25 1.1 Fingerprint sensor 

FIG. • 5 shows an example of the fingerprint sensor 80. In FIG. 5, the 
fingerprint sensor 80 includes M (M is an integer of two or more) power supply lines 
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200 and N (N is an integer of two or more) output lines 202. A capacitance detection 
element 204 is provided at each intersecting point of the M power supply lines 200 and 
the N output lines 202. The capacitance detection element 204 shown in FIG 5 is 
illustrated as a closed circuit when a finger is in contact with the capacitance detection 
5 element 204. The capacitance detection element 204 includes a variable capacitor C F 
of which the capacitance is changed depending on a ridge/valley pattern of a finger, and 
a signal amplification element such as a signal amplification MIS thin film 
semiconductor device (hereinafter abbreviated as "signal amplification TFT") 206. If 
a finger is not in contact with the capacitance detection element 204, a grounding 
10 terminal of the variable capacitor Cf is in an open state. The variable capacitor Cf is 
described later. 

The M power supply lines 200 are connected with drains D of the N signal 
amplification TFTs 206 arranged along the corresponding row. The M power supply 
lines 200 are connected with a common power supply line 212 through M power supply 

15 pass gates 210. Specifically, the power supply pass gate 210 is formed by using a MIS 
thin film semiconductor device. A source S of the power supply pass gate 210 is 
connected with the power supply line 200, and a drain D of the power supply pass gate 
210 is connected with the common power supply line 212. A power supply shift 
register 222 is provided to a power supply select circuit 220 in addition to the M power 

20 supply pass gates 210 and the common power supply line 212. A gate G of each of the 
M power supply pass gates 210 is connected with a power supply select output line 224 
of the power supply shift register 222. 

The N output lines 202 are connected with sources S of the N signal 
amplification TFTs 206 arranged along the corresponding column. The N output lines 

25 202 are connected with a common output line 232 through N output signal pass gates 
230. Specifically, the output signal pass gate 230 is formed by using an MIS thin film 
semiconductor device. A drain D of the output signal pass gate 230 is connected with 
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the output line 202, and a source S of the output signal pass gate 230 is connected with 
the common output line 232. An output signal shift register 242 is provided to an 
output signal select circuit 240 in addition to the N output signal pass gates 230 and the 
common output line 232. A gate G of the output signal pass gate 230 is connected 
5 with an output select output line 244 of the output signal shift register 242. 

FIG. 6 is a cross-sectional view showing the capacitance detection element 204 
shown in FIG. 5. A finger is not in contact with the capacitance detection element 204 
in this figure. The capacitance detection element 204 includes a signal detection 
element 208 in addition to the signal amplification TFT 206 which is the signal 
10 amplification element. 

In FIG. 6, a semiconductor film 252 including a source region 252A, a drain 
region 252B, and a channel region 252C present between the source region 252A and 
the drain region 252B is formed on an insulating layer 250. A gate insulating film 254 
is formed on the semiconductor film 252. A gate electrode 256 is formed in a region 
15 which faces the channel region 252C with the gate insulating film 254 interposed 
therebetween. The semiconductor film 252, the gate insulating film 254, and the gate 
electrode 256 make up the signal amplification TFT 206. The power supply pass gate 
210 and the output signal pass gate 230 are formed in the same manner as the signal 
amplification TFT 206. 

20 The signal amplification TFT 206 is covered with a first interlayer dielectric 

260. A first interconnect layer 262 corresponding to the output line 202 shown in FIG. 
5 is formed on the first interlayer dielectric 260. The first interconnect layer 262 is 
connected with the source region 252A of the signal amplification TFT 206. 

The first interconnect layer 262 is covered with a second interlayer dielectric 

25 264. A second interconnect layer 266 corresponding to the power supply line 200 
shown in FIG. 5 is formed on the second interlayer dielectric 264. The second 
interconnect layer 266 is connected with the drain region 252B of the signal 
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amplification TFT 206. As another structure differing from the structure shown in FIG. 
6, the second interconnect layer 266 may be formed on the first interlayer dielectric 260, 
and the first interconnect layer 262 may be formed on the second interlayer dielectric 
264. 

5 A capacitance detection electrode 270 is formed on the second interlayer 

dielectric 264. A capacitance detection dielectric film 272 is formed to cover the 
capacitance detection electrode 270. The capacitance detection dielectric film 272 is 
located on the outermost surface of the fingerprint sensor 80 and functions as a 
protective film. A finger comes in contact with the capacitance detection dielectric 
10 film 272. The signal detection element 208 is made up of the capacitance detection 
electrode 270 and the capacitance detection dielectric film 272. 

1.1.1 Fingerprint detection operation 

A fingerprint is detected by allowing a finger to come in contact with the 
15 capacitance detection dielectric film 272 shown in FIG. 6. A start switch 
(pressure-sensitive switch, for example) 42 of the fingerprint sensor 80 is operated to 
allow a power supply inside the input device 68 to be operated, whereby power is 
automatically supplied to the fingerprint sensor 80. The input device 68 may be 
provided to the personal computer 84, and power may be supplied from a power supply 
20 section of the personal computer 84. 

In this embodiment, a signal is sequentially removed from the M X N 
capacitance detection elements 204 by providing a power supply voltage to one of the 
M power supply lines 200 shown in FIG. 5 and detecting a signal from one of the N 
output lines 202. 

25 The fingerprint detection operation is roughly divided into (1) a case where a 

ridge (projecting section) of the fingerprint pattern comes in contact with the 
capacitance detection dielectric film 272, and (2) a case where a valley (recess section) 
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of the fingerprint pattern faces the capacitance detection dielectric film 272. 

(1) Case where ridge (projecting section) of fingerprint pattern comes in 
contact with capacitance detection dielectric film 272 
5 FIG. 7 shows an equivalent circuit of the capacitance detection element 204 in 

this case. A symbol 300 corresponds to a ridge of a human fingerprint. A grounding 
electrode 300 which faces the capacitance detection electrode 270 shown in FIG. 6 with 
the dielectric film 272 interposed therebetween is formed in a region indicated by the 
symbol 300. A power supply voltage Vdd is supplied from the common power supply 

10 line 212. A symbol C T indicates a transistor capacitor of the signal amplification TFT 
206. A symbol C D indicates a capacitor between the detection electrode 270 and the 
grounding electrode (finger) 300. 

The length of the gate electrode of the signal amplification TFT 206 is referred 
to as L (nm), the width of the gate electrode is referred to as W (^im), the thickness of 

15 the gate insulating film is referred to as tox (|^m) 9 the relative dielectric constant of the 
gate insulating film is referred to as eox, and the dielectric constant under vacuum is 
referred to as eo. The capacitance of the transistor capacitor C T is expressed by the 
following equation (6). 

20 Ct ~ co - eox • L • W/tox (6) 

The area of the capacitance detection electrode 270 is referred to as S (jam 2 ), 
the thickness of the capacitance detection dielectric film 272 is referred to as td (^im), 
and the relative dielectric constant of the capacitance detection dielectric film 272 is 
25 referred to as ed. - The capacitance of the capacitor C D is expressed by the following 
equation (7). 
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C D = eo • ed • S/td 



(7) 



In the equivalent circuit shown in FIG. 7, a voltage Vgt applied to the gate of 
the signal amplification TFT 206 is expressed as follows. 



If the capacitance of the capacitor Co is set sufficiently greater than the 
capacitance of the transistor capacitor C T (C D > 10 X Or, for example), the 
denominator in the equation (8) becomes infinite, whereby V G t is approximated as 
follows. 



As a result, the signal amplification TFT 206 is in an off state since almost no 
voltage is applied to the gate of the signal amplification TFT 206. Therefore, a current 
I which flows between the source and the drain of the signal amplification TFT 206 is 
extremely decreased. Therefore, the measurement point can be judged to be the ridge 
(projecting section) of the fingerprint pattern by measuring the current I. 

(2) Case where valley (concave section) of fingerprint pattern faces capacitance 
detection dielectric film 272 

FIG. 8 shows an equivalent circuit of the capacitance detection element 204 in 
this case. A symbol 302 corresponds to a valley of a human fingerprint. In this case, 
a capacitor C A in which air is a dielectric is formed between the dielectric film 272 and 
the valley of the fingerprint in addition to the capacitor C D shown in FIG. 7. 

In the equivalent circuit shown in FIG 8, a voltage Vgv applied to the gate of 



V GT = Vdd/(l +Cd/C t ) 



(8) 



V GT - 0 



(9) 
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the signal amplification TFT 206 is expressed as follows. 

V GV = Vdd/{[l +(1/C T )] X 1/[(1/C d ) + (1/Ca)]} (10) 

5 If the capacitance of the capacitor Co is set sufficiently greater than the 

capacitance of the transistor capacitor Or (Co > 10 X Cj, for example), the equation 
(10) is approximated as follows. 

V GV = Vdd/[1 + (Ca/Ct)] (11) 

10 

If the capacitance of the transistor capacitor C T is set sufficiently greater than 
the capacitance of the capacitor C A formed by the valley of the fingerprint (C T > 10 X 
Ca, for example), the equation (11) is approximated as follows. 

15 V GV ^ Vdd (12) 

As a result, the signal amplification TFT 206 is in an on state since the power 
supply voltage Vdd is applied to the gate of the signal amplification TFT 206. 
Therefore, the current I which flows between the source and the drain of the signal 
20 amplification TFT 206 is extremely increased. Therefore, the measurement point can 
be judged to be the valley (recess section) of the fingerprint pattern by measuring the 
current I. 

The variable capacitor Cf shown in FIG. 5 has a capacitance equal to the 
capacitance of the capacitor Co when the ridge of the fingerprint is in contact with the 
25 capacitance detection dielectric film 272, and has a capacitance equal to the sum of the 
capacitance of the capacitor Co and the capacitance of the capacitor C A when the valley 
of the fingerprint faces the capacitance detection dielectric film 272. Therefore, the 
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capacitance of the variable capacitor Cf varies corresponding to the ridge and valley of 
the fingerprint. The ridge or valley of the fingerprint can be detected by detecting the 
current based on the change in capacitance corresponding to the ridge and valley of the 
fingerprint. 

5 A fingerprint pattern can be detected by carrying out the above-described 

operation in each of the M X N capacitance detection elements 204 by time division. 
In more detail, the ridge or valley of the fingerprint is sequentially detected in the 
capacitance detection elements located in each column in the first row, and the ridge or 
valley of the fingerprint is then detected in the second row. The ridge or valley of the 

10 fingerprint is detected in pixel units in this manner. This enables a fingerprint image 
as shown in FIGS. 9A and 9B to be obtained, for example. In this embodiment, 
fingerprint images are periodically captured by using the fingerprint sensor 80. 

In the case where a positive power supply is used as the power supply voltage 
Vdd, the signal amplification TFT 206 is formed by using an enhancement N-type 

15 transistor in which a drain current does not flow at a gate voltage of about zero. 
Provided that the gate voltage at which the drain current is minimum (minimum gate 
voltage) in the transfer characteristics of the signal amplification TFT 206 is Vmin, Co 

> 10 X C T is satisfied by satisfying 0 < Vmin < 0.1 X Vdd. 

In the case where a negative power supply is used as the power supply voltage 
20 Vdd, the signal amplification TFT 206 is formed by using an enhancement P-type 
transistor in which a drain current does not flow at a gate voltage of about zero. 
Provided that the gate voltage at which the drain current is minimum (minimum gate 
voltage) in the transfer characteristics of the signal amplification TFT 206 is Vmin, Co 

> 10 X C T is satisfied by satisfying 0.1 X Vdd < Vmin < 0. 

25 In this embodiment, the control information is output by using thus captured 

fingerprint image in this manner. In this case, the processing load is reduced by using 
feature points (minutiae points) of the captured fingerprint image. 
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FIGS. 9 A and 9B show examples of feature (or minutiae) points of a 
fingerprint. FIG. 9A shows ridge bifurcations of a fingerprint. FIG. 9B shows ridge 
endings of a fingerprint. Ridge bifurcations, for example, are extracted from a 
fingerprint image captured by the fingerprint sensor 80. In FIGS. 9A and 9B, the 
5 fingerprint image shows ridges (projections) of the fingerprint. A ridge bifurcation is a 
portion at which a ridge divides into two or more branches. A ridge ending is a portion 
at which a ridge ends. 

Since forms of fingerprints are not identical, distribution of ridge bifurcations 
or ridge endings of a fingerprint differs between individuals. Therefore, if ridge 
10 bifurcations or ridge endings can be determined, it suffices to merely compare the 
distribution of the ridge bifurcations or the ridge endings. This reduces the amount of 
information to be compared, whereby the load of comparison processing can be 
reduced. 

15 1 .2 Operation flow 

FIG. 10 shows a processing flow of the input device in this embodiment. A 
program for executing the processing shown in FIG. 10 is, stored in the ROM 74 or 
RAM 76. The CPU 72 performs processing according to the program. 

The CPU 72 initializes each variable before performing the processing as the 
20 input device (step S400). 

Whether or not a finger (or an object to be detected) of a user is in contact with 
the detection surface of the fingerprint sensor 80 is judged (step S401). If it is 
determined that no finger is in contact with the detection surface (step S401: N), the 
operation is returned to the step S400. 
25 If it is determined that a finger is in contact with the detection surface (step 

S401: Y), a fingerprint image is captured (step S402). The captured fingerprint image 
for one frame is stored in the RAM 76. 
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The CPU 72 reads the fingerprint image for one frame stored in the RAM 76, 
and calculates the center of gravity of the fingerprint image (step S403). The center of 
gravity is the center of a region including a fingerprint pattern in the captured 
fingerprint image. The center of gravity is calculated as the center of the fingerprint 
image obtained by integral operations in the X axis direction and in the Y axis direction, 
for example. 

Provided that the coordinates of the center of gravity are (Xg,Yg) 5 the center of 
gravity is calculated as shown by the following equations. 



X G =YiF(i)/S 



M 



N M 



(13) 



(14) 



(15) 



S is the area of the fingerprint image, F(i) is the number of dots detected on the 
output line i, and G(j) is the number of dots detected on the power supply line j. 

The center of gravity of the fingerprint image in the current frame is compared 
with the center of gravity of the fingerprint image in the preceding frame (step S404). 
The component X and the component Y of the difference between the center of gravity 
of the fingerprint image in the current frame and the center of gravity of the fingerprint 
image in the preceding frame are output as the moving amount AX1 of the center of 
gravity in the X axis direction and the moving amount AY1 of the center of gravity in 
the Y axis direction, respectively. The center of gravity of the fingerprint image in the 
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preceding frame has been stored in the RAM 76. The center of gravity of the 
fingerprint image in the preceding frame is updated by the center of gravity of the 
fingerprint image in the current frame calculated in the step S403 after the comparison 
processing in the step S404. In this example, the fingerprint image in the preceding 
5 frame is used as the object for comparison. However, the present invention is not 
limited thereto. The fingerprint image in a frame at least two frames before the current 
frame (image of the object to be detected in a frame at least two frames before the 
current frame) may be used as the object for comparison. 

The area of the captured fingerprint image is calculated (step S405). The area 

10 of the fingerprint image may be calculated by the above integral operation. The area 
of the fingerprint image may be calculated as an equal value by the number of dots 
(pixels) detected as the ridge or valley of the fingerprint image. 

The area of the fingerprint image thus calculated is compared with the area of 
the fingerprint image in the preceding frame (step S406). The moving amount Az of 

15 the finger of the user in the Z axis direction is calculated corresponding to the change in 
the area between the fingerprint images before and after movement as shown by the 
equation (5). The area of the fingerprint image in the preceding frame has been stored 
in the RAM 76. The area of the fingerprint image in the preceding frame is updated by 
the area of the fingerprint image in the current frame calculated in the step S405 after 

20 the comparison processing in the step S406. In this example, the fingerprint image in 
the preceding frame is used as the object for comparison. However, the present 
invention is not limited thereto. The fingerprint image in a frame at least two frames 
before the current frame (image of the object to be detected in a frame at least two 
frames before the current frame) may be used as the object for comparison. 

25 FIGS. 1 1 A, 1 IB, and 1 1 C show changes in a fingerprint image when a finger is 

moved in the Z axis direction. A fingerprint image 452 in the frame f shown in FIG. 
11B is the original position in a fingerprint image detection region 450. In the case 
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where a fingerprint image captured in the frame (f + 1) is a fingerprint image 454 shown 
in FIG. 11 A, the user has moved the finger in the direction in which the finger is pulled 
up from the detection surface of the fingerprint sensor 80 ((-) direction of the Z axis). 
Since the finger is moved in the direction in which the pressure applied to the detection 
5 surface by the finger is decreased, the area in which the finger is in contact with the 
detection surface is decreased. In the case where the fingerprint image captured in the 
frame (f + 1) is a fingerprint image 456 shown in FIG. 11C, the user has moved the 
finger in the direction in which the finger is pressed against the detection surface of the 
fingerprint sensor 80 ((+) direction of the Z axis). Since the finger is moved in the 

10 direction in which the pressure applied to the detection surface by the finger is increased, 
the area in which the finger is in contact with the detection surface is increased. 
Therefore, the moving direction and the moving amount of the finger in the Z axis 
direction can be detected by associating the change in the area of the fingerprint image 
with the moving amount Az of the finger in the Z axis direction. 

15 The area of the captured fingerprint image can be calculated by using the 

fingerprint sensor 80 having the configuration shown in FIG. 5. 

FIG. 12 is a diagram for illustrating a method of reducing processing load for 
calculating an area of a fingerprint image. A fingerprint image 458 is captured by 
allowing the fingerprint sensor 80 shown in FIG. 5 to scan in the fingerprint image 

20 detection region 450 in the X axis direction and the Y axis direction. In this case, the 
number Oc of output lines through which the ridge or valley of the fingerprint is 
detected can be specified in the X axis direction by an output line Ol at which detection 
of the ridge or valley of the fingerprint is started and an output line 02 at which the 
ridge or valley of the fingerprint is detected last. The number Dc of power supply 

25 lines through which the ridge or valley of the fingerprint is detected can be specified in 
the Y axis direction by a power supply line Dl at which detection of the ridge or valley 
of the fingerprint is started and a power supply line D2 at which the ridge or valley of 
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the fingerprint is detected last. Therefore, a value Sr equivalent to the area of the 
fingerprint image 458 can be calculated by the number Oc of output lines and the 
number Dc of power supply lines. 



5 Oc = 02-01 + 1 (16) 

Dc = D2-Dl + 1 (17) 

Sr = Oc . Dc (18) 

10 



Since the rectangle circumscribed on the outline of the captured fingerprint 
image is easily calculated by specifying the power supply lines and the output lines of 
the fingerprint sensor 80, the processing load for calculating the area of the fingerprint 
image can be reduced. 

15 Moreover, the center of gravity of the fingerprint image 458 can be specified in 

the step S403 based on the intermediate position of the output lines Ol to 02 and the 
intermediate position of the power supply lines Dl to D2. This enables the processing 
load for calculating the center of gravity of the fingerprint image 458 to be significantly 
reduced. Provided that the coordinates of the center of gravity are (X M? Y M ), the center 

20 of gravity is calculated as shown by the following equations. 

X M = (01 +02)/2 (19) 
Y M = (D1 +D2)/2 (20) 

25 

The description is given with reference to FIG. 10. After the moving amount 
Az in the Z axis direction is calculated in the step S406, feature points (minutiae points) 
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of the fingerprint image is extracted (step S407). Feature points used herein refer to 
ridge endings or ridge bifurcations of a fingerprint. Distribution of these feature points 
is then calculated. This enables movement of the fingerprint image on the detection 
surface to be detected by the movement of the feature points (or distribution of the 
5 feature points). 

The feature points of the fingerprint image calculated in the step S407 is 
compared with the feature points of a fingerprint image in the preceding frame (step 
S408). The feature points (distributions of the feature points) of the fingerprint image 
in the current frame and the preceding frame are compared, and the moving amount 

10 AX2 in the X axis direction, the moving amount AY2 in the Y axis direction, and the 
rotation angle p around the Z axis are calculated so that the similarity of the feature 
points (distributions of the feature points) is maximized. 

FIGS. 13A and 13B schematically show comparing processing of the feature 
points. In FIGS. 13A and 13B, feature points Pf, Qf, and R f extracted from the 

15 fingerprint image in the frame f are moved to positions of feature points Pf+i, Qf+i, and 
Rf+i of the fingerprint image captured in the frame (f+1). As shown in FIG. 13A ? the 
CPU 72 calculates the moving amount AX2 in the X axis direction and the moving 
amount AY2 in the Y axis direction so that at least the feature points Pf, Q f , and Rf 
among three or more extracted feature points respectively coincide with the 

20 corresponding feature points Pf+i, Qf+i, and Rf+i. As shown in FIG. 13B, the CPU 72 
calculates a reference point e and the rotation angle P around the Z axis with respect to 
the reference point e as the center so that at least the feature points Pf, Qf, and Rf among 
three or more extracted feature points respectively coincide with the corresponding 
feature points Pf+i, Qf+i, and Rf+i. 

25 FIGS. 14A, 14B, and 14C show changes in a fingerprint image when a finger is 

moved around the Z axis. A fingerprint image 462 shown in FIG. 14A corresponds to 
the case where the finger is rotated around the Z axis in the (-) direction in the 
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fingerprint image detection region 450 with respect to a fingerprint image 460 in the 
frame f shown in FIG. 14B. A fingerprint image 464 shown in FIG 14C corresponds 
to the case where the finger is rotated around the Z axis in the (+) direction. 

Therefore, the changes in the fingerprint image shown in FIGS. 14 A, 14B, and 

5 14C can be calculated as the rotation angle p around the Z axis by performing the 
comparing processing shown in FIG. 13B, for example. 

The description is given with reference to FIG. 10. If the movement vector of 
the fingerprint image (the moving direction and the moving amount of the feature point 
of the fingerprint image in the X axis direction and the Y axis direction) is calculated in 

10 the step S408, the CPU 72 calculates the rotation angle a around the X axis according to 
the equation (4), and calculates the rotation angle y around the Y axis according to the 
equation (3). The CPU 72 calculates the moving amount Ax in the X axis direction 
according to the equation (1), and calculates the moving amount Ay in the Y axis 
direction according to the equation (2) (step S409). If the processing is finished (step 

15 S4 1 0: Y) 3 processing is terminated (END). If the processing is not finished (step S4 1 0: 
N), the operation is returned to the step S401. 

FIGS. 15 A, 15B, and 15C show changes in a fingerprint image when a finger is 
moved around the Y axis. A user rotates the finger around the Y axis while sliding the 
finger on the detection surface of the fingerprint sensor 80 in these figures. Therefore, 

20 the center of gravity of the fingerprint image is changed only to a small extent in FIGS. 
15 A, 15B, and 15C. 

A fingerprint image 472 shown in FIG. 1 5 A corresponds to the case where the 
finger is rotated around the Y axis in the (-) direction in the fingerprint image detection 
region 450 with respect to a fingerprint image 470 in the frame f shown in FIG. 15B. 
25 Specifically, the feature point of the fingerprint image is moved in the (+) direction of 
the X axis while the center of gravity of the fingerprint image is changed only to a small 
extent. A fingerprint image 474 shown in FIG. 15C corresponds to the case where the 
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finger is rotated around the Y axis in the (+) direction. Specifically, the feature point 
of the fingerprint image is moved in the (-) direction of the X axis while the center of 
gravity of the fingerprint image is changed only to a small extent. In this case, the 
rotation angle y corresponding to the moving amount of the feature point of the 
5 fingerprint image shown in FIGS. 15 A, 15B, and 15C is calculated according to the 
equation (4). 

FIGS. 16 A, 16B, and 16C show changes in a fingerprint image when a finger is 
rotated around the X axis. A user rotates the finger around the X axis while sliding the 
finger on the detection surface of the fingerprint sensor 80 in these figures. Therefore, 
10 the center of gravity of the fingerprint image is changed only to a small extent in FIGS. 
16A ? 16B, and 16C. 

A fingerprint image 482 shown in FIG. 16A corresponds to the case where the 
finger is rotated around the X axis in the (-) direction in the fingerprint image detection 
region 450 with respect to a fingerprint image 480 in the frame f shown in FIG. 16B. 

15 A fingerprint image 484 shown in FIG. 16C corresponds to the case where the finger is 
rotated around the X axis in the (+) direction. Specifically, the feature point of the 
fingerprint image is moved in the X axis direction while the center of gravity of the 
fingerprint image is changed only to a small extent. In this case, the rotation angle a 
corresponding to the moving amount of the feature point of the fingerprint image shown 

20 in FIGS. 16A, 16B, and 16C is calculated according to the equation (3). 

FIGS. 17A, 17B, and 17C show changes in a fingerprint image when a finger is 
moved in the X axis direction. A user moves the finger in the X axis direction while 
sliding the finger on the detection surface of the fingerprint sensor 80 in these figures. 
Therefore, the movement vector of the center of gravity of the fingerprint image almost 

25 coincides with the movement vector of the feature point of the fingerprint image in 
FIGS. 17A, 17B, and 17C. 

A fingerprint image 492 shown in FIG. 1 7A corresponds to the case where the 
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finger is moved in the (-) direction of the X axis direction with respect to a fingerprint 
image 490 in the frame f shown in FIG 1 7B in the fingerprint image detection region 
450. A fingerprint image 494 shown in FIG 17C corresponds to the case where the 
finger is moved in the (+) direction of the X axis direction. In this case, the moving 
5 amount Ax in the X axis direction almost coincides with the moving amount AX2 of the 
feature point of the fingerprint image. However, a value to which the rotation angle y 
around the Y axis is added is actually calculated as shown in the equation (1). 

FIGS. 1 8A, 1 8B, and 1 8C show changes in a fingerprint image when a finger is 
moved in the Y axis direction. A user moves the finger in the Y axis direction while 

10 sliding the finger on the detection surface of the fingerprint sensor 80 in these figures. 
Therefore, the movement vector of the center of gravity of the fingerprint image almost 
coincides with the movement vector of the feature point of the fingerprint image in 
FIGS. 18A, 18B, and 18C. 

A fingerprint image 502 shown in FIG 18A corresponds to the case where the 

15 finger is moved in the (-) direction of the Y axis direction in the fingerprint image 
detection region 450 with respect to a fingerprint image 500 in the frame f shown in FIG. 
18B. A fingerprint image 504 shown in FIG. 18C corresponds to the case where the 
finger is moved in the (+) direction of the Y axis direction. In this case, the moving 
amount Ay in the Y axis direction almost coincides with the moving amount AY2 of the 

20 feature point of the fingerprint image. However, a value to which the rotation angle a 
around the X axis is added is actually calculated as shown in the equation (2). 

According to the input device in this embodiment, the control information in 
the six-axis directions can be generated by using a capture image of the object to be 
detected such as a fingerprint. This enables the provision of an input device enabling 

25 the control instruction (movement instruction, for example) in the six-axis directions. 
In particular, an input device which is highly sensitive, extremely small, and extremely 
lightweight, consumes only a small amount of electric power, and enables the control 
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instruction in the six-axis directions can be provided by using the capacitance type 
fingerprint sensor 

Modification example 

5 In the above embodiment, the moving amount Ax and the moving amount Ay 

of the object to be detected on the detection surface are detected by using the movement 
(AX2 and AY2) of the feature point of the fingerprint image. However, the present 
invention is not limited thereto. 

For example, the difference (AXg and AYg) between the center of gravity of 

10 the fingerprint image of the object to be detected in the current frame and the center of 
gravity of the fingerprint image of the object to be detected in a frame at least one frame 
before the current frame may be calculated by using the X coordinate (Xg) and the Y 
coordinate (Yg) of the center of gravity calculated by using the equations (13) to (15) as 
the center of gravity of the fingerprint image. In this case, the moving amount Ax in 

15 the X axis direction and the moving amount Ay in the Y axis direction of the object to 
be detected on the detection surface may be expressed by the following equations 
instead of the equations (1) and (2). 

Ax = AXc-a.y (21) 

20 

Ay- AY G - b. a (22) 

The difference (AXg and AYg) between the center of gravity of the fingerprint 
image of the object to be detected in the current frame and the center of gravity of the 
25 fingerprint image of the object to be detected in a frame at least one frame before the 
current frame may be calculated by using the X coordinate (Xg) and the Y coordinate 
(Yg) of the center of gravity calculated by using the equations (19) and (20) as the 



32 



EP-64380US 

center of gravity of the fingerprint image. In this case, the moving amount Ax in the X 
axis direction and the moving amount Ay in the Y axis direction of the object to be 
detected on the detection surface may be expressed by the following equations instead 
of the equations (1) and (2). 

5 

Ax = AX M -a.y (23) 

Ay = A Ym — b • a (24) 

10 In both cases, the control information in the six-axis directions can be 

generated in the same manner as in the above embodiment. In particular, an input 
device which is highly sensitive, extremely small, and extremely lightweight, consumes 
only a small amount of electric power, and enables the control instruction in the six-axis 
directions can be provided by using the capacitance type fingerprint sensor. 

15 

2. Information Device 

FIG. 19 shows a block diagram showing configuration of an IC card to which 
the input device according to this embodiment is applied. An IC card 600 includes an 
input device 610 using the above-described fingerprint sensor, an image generation 

20 section (processing section which performs control processing of a predetermined object 
of control in a broad sense) 620, and a display section 630. The input device 610 is the 
input device described with reference to FIG. 1 or 4. The image generation section 620 
is realized by a CPU and a software program stored in a ROM or RAM. The display 
section 630 is realized by an LCD panel and a driver circuit of the LCD panel. 

25 The image generation section 620 generates image data (performs control 

processing in a broad sense) based on the control information output from the input 
device 610. In more detail, the image generation section 620 generates image data of 
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an image which is changed corresponding to the movement instruction in the six-axis 
directions by the input device 610. The display section 630 displays an image based 
on the image data generated by the image generation section 620. 

In the IC card 600 having such a configuration, a pointer displayed in the 
5 display section 630 can be moved or an image displayed in the display section 630 can 
be scrolled by allowing the user to instruct the movement by moving the fingerprint 
image of the finger in the six-axis directions in the input device 600. 

The above description illustrates the case where the IC card is used as an 
information device. However, the input device according to this embodiment may be 
10 applied to a PDA, a portable telephone, a three-dimensional CAD device, a virtual 
reality experience device, an electronic musical instrument, or the like. 

The present invention is not limited to the above-described embodiment. 
Various modifications can be made within the scope of the present invention. 

The above embodiment illustrates the input device using the fingerprint sensor. 
15 However, the present invention is not limited thereto. The control information may be 
output in the same manner as described above by capturing an image of a 
two-dimensional or three-dimensional object other than a fingerprint. The present 
invention may also be applied to an input device which does not include a detection 
surface. 

20 This embodiment illustrates the case where the control information in the 

six-axis directions is output corresponding to the difference between the current frame 
and the preceding frame (frame at least one frame before the current frame). However, 
the present invention is not limited thereto. For example, the difference from the 
original position as the reference point may be added in each frame, and the absolute 

25 value of the difference may be output as the control information in the six-axis 
directions. 

Part of requirements of any claim of the present invention could be omitted 
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from a dependent claim which depends on that claim. Moreover, part of requirements 
of any independent claim of the present invention could be made to depend on any other 
independent claim. 

Moreover, there can be provided embodiments of the present invention having 
5 features as follows. 

According to one embodiment of the present invention, there is provided an 
input device which generates control information by moving an object to be detected, 
comprising: 

an image capture section which captures an image of the object to be detected; 
10 a feature point extraction section which extracts a feature point of the image 

captured by the image capture section; 

a difference calculation section which calculates a difference between a 
reference position and a position of the feature point; and 

a control information output section which outputs the control information 
15 corresponding to the calculated difference. 

In this input device, feature points of an image of the moving object to be 
detected are extracted, and the control information corresponding to the difference 
between the reference position and the position of the feature point is output. 
Therefore, it suffices to merely perform comparison processing using the feature points. 
20 This reduces the processing load of the input device which outputs the control 
information in response to the movement of the image of the object to be detected. 

In this input device, the reference position may be a position of a feature point 
extracted from an image of the object to be detected in a frame at least one frame before 
the current frame. 

25 According to this input device, since an image of the object to be detected in a 

frame at least one frame before the current frame is used, the image can be captured in a 
frame cycle, and the control information can be output in frame units. This enables the 
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control information can be generated with higher accuracy. 

According to another embodiment of the present invention, there is provided an 
input device which generates control information by moving an object to be detected, 
comprising: 

5 an image capture section which captures an image of the object to be detected; 

an area calculation section which calculates an area of the image captured by 
the image capture section; 

a difference calculation section which calculates a difference between an area 
of the image of the object to be detected in a frame at least one frame before the current 
10 frame and the area calculated by the area calculation section; and 

a control information output section which outputs the control information 
corresponding to the calculated difference. 

According to this input device, the difference in the area of the captured image 
of the object to be detected between the frames is calculated, and the control 
15 information corresponding to the difference is output. Therefore, the control 
instruction in another axis direction which is not limited to the movement of the 
position of the captured image can be performed. This enables operability to be 
further improved. 

In this input device, the image capture section may include a detection surface 
20 and capture an image of an object to be detected which is in contact with the detection 
surface; and the control information output section may output the control information 
corresponding to the moving amount of the object to be detected in an axis direction 
perpendicular to the detection surface. 

According to this input device, since the moving distance of the object to be 
25 detected in the axis (Z axis) direction perpendicular to the detection surface can be 
specified by the change in the area of the image of the object to be detected which is in 
contact with the detection surface, the control information (Az) corresponding not only 
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to the movement in the first and second axis directions which are perpendicular to each 
other on the detection surface, but also to the movement in the axis direction 
perpendicular to the first and second axis directions can be generated. This enables 
operability to be further improved. 
5 According to further embodiment of the present invention, there is provided an 

input device which generates control information by moving an object to be detected, 
comprising 

an image capture section which includes a detection surface and captures an 
image of the object to be detected which is in contact with the detection surface; 
10 a center-of-gravity calculation section which calculates a center of gravity of 

the image captured by the image capture section; 

a feature point extraction section which extracts a feature point of the captured 

image; 

a center-of-gravity movement detection section which calculates a first 
15 difference (AX1 and AY1) between a center of gravity of an image of the object to be 
detected in a frame at least one frame before the current frame and the center of gravity 
calculated by the center-of-gravity calculation section; 

a feature point position movement detection section which calculates a second 
difference (AX2 and AY2) between a position of a feature point of an image of the 
20 object to be detected in a frame at least one frame before the current frame and a 
position of the feature point extracted by the feature point extraction section; and 

a rotation angle detection section which detects a rotation angle around one of 
a first axis and a second axis (X axis and Y axis) which are perpendicular to each other 
on the detection surface based on the first difference and the second difference, 
25 wherein the rotation angle detection section calculates the rotation angle (y) 

around the second axis (Y axis) by subtracting a first axis direction component (AX2) of 
the second difference from a first axis direction component (AX1) of the first difference, 
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calculates the rotation angle (a) around the first axis (X axis) by subtracting a second 
axis direction component (AY2) of the second difference from a second axis direction 
component (AY1) of the first difference, and outputs the control information 
corresponding to the rotation angle around the first axis or the second axis. 

5 In this input device, the rotation angle around the first axis is calculated by 

subtracting the second axis direction component (AY2) of the second difference from 
the second axis direction component (AY1) of the first difference, and the rotation angle 
around the second axis is calculated by subtracting the second axis direction component 
(AY2) of the second difference from the second axis direction component (AY1) of the 

10 first difference. This makes it possible to generate the control information (a and 7) 
around the first axis and the second axis relating to the three-dimensional movement 
(rotation) of the object to be detected from the two-dimensional image of the object to 
be detected without increasing the processing load. Therefore, an input device which 
is capable of further improving operability can be provided. 

15 In this input device, the rotation angle detection section may calculate a 

rotation angle between a feature point of an image of the object to be detected in a 
frame at least one frame before the current frame and the feature point extracted by the 
feature point extraction section around a reference point as a rotation angle around a 
third axis which is perpendicular to the detection surface, and output the control 

20 information corresponding to the rotation angle around the third axis. 

According to this input device, since the rotation angle of the feature point of 
the image of the object to be detected around the reference point can be calculated, the 
control information (p) corresponding to the rotation angle around the third axis 
perpendicular to the detection surface can be generated while reducing the processing 

25 load. Therefore, an input device which is capable of further improving operability can 
be provided. 

In this input device, the control information (Ax) corresponding to the 
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movement in the first axis (X axis) direction may be output by subtracting a value 
obtained by weighting the rotation angle (y) around the second axis (Y axis) with a first 
coefficient (a) from the first axis direction component (AX2) of the second difference; 
and the control information (Ay) corresponding to the movement in the second axis (Y 
5 axis) direction may be output by subtracting a value obtained by weighting the rotation 
angle (a) around the first axis (X axis) with a second coefficient (b) from the second 
axis direction component (AY2) of the second difference. 

In this input device, the control information (Ax) corresponding to the 
movement in the first axis (X axis) direction may be output by subtracting a value 
10 obtained by weighting the rotation angle (y) around the second axis (Y axis) with a first 
coefficient (a) from the first axis direction component (AX G or AX M ) of the first 
difference, and the control information (Ay) corresponding to the movement in the 
second axis (Y axis) direction may be output by subtracting a value obtained by 
weighting the rotation angle (a) around the first axis (X axis) with a second coefficient 
15 (b) from the second axis direction component (AY G or AY M ) of the first difference. 

According to this input device, since the movement of the object to be detected 
in the first and second axis directions is detected by adding the rotation angle around 
each axis to the movement of the feature point or the center of gravity of the image of 
the object to be detected, optimum movement in the first and second axis directions can 
20 be detected corresponding to the manner of the movement of the object to be detected 
(user's peculiar operation for moving the object to be detected and the like). This 
enables the control information in the first or second axis direction to be output with 
high accuracy in the case of outputting the control information around the first or 
second axis, whereby an input device which is capable of further improving operability 
25 can be provided. 

In this input device, the object to be detected may be a fingerprint. 

According to this input device, since an extremely small and lightweight 
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fingerprint sensor can be used, an input device which is capable of further improving 
operability can be mounted on a portable information device. 

According to still another embodiment of the present invention, there is 
provided an information device comprising any of the above input devices; and a 
5 processing section which performs control processing based on the control information 
from the input device. 

According to this information device, an information device which is capable 
of further improving operability can be provided. 

According to yet another embodiment of the present invention, there is 
10 provided a control information generation method of generating control information by 
using a captured image of an object to be detected, the control information generation 
method comprising: 

extracting a feature point of an image of the object to be detected; 

calculating a difference between a reference position and a position of the 
15 feature point; and 

outputting the control information corresponding to the difference. 

According to further embodiment of the present invention, there is provided a 
control information generation method of generating control information by using a 
captured image of an object to be detected, the control information generation method 
20 comprising: 

calculating an area of the image of the object to be detected which is in contact 
with a detection surface; 

calculating a difference between the calculated area and an area of an image of 
the object to be detected in a frame at least one frame before the current frame; and 
25 outputting the control information corresponding to the difference in response 

to the movement of the object to be detected in a direction perpendicular to the 
detection surface. 
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According to still further embodiment of the present invention, there is 
provided a control information generation method of generating control information by 
using a captured image of an object to be detected, the control information generation 
method comprising: 

5 calculating a center of gravity of the image of the object to be detected which is 

in contact with a detection surface and extracting a feature point of the image; 

calculating a difference between the center of gravity and a center of gravity of 
an image of the object to be detected in a frame at least one frame before the current 
frame and calculating a difference between a position of the feature point and a position 
10 of a feature point of an image of the object to be detected in the frame at least one frame 
before the current frame; 

detecting a rotation angle around one of a first axis and second axis which are 
perpendicular to each other on the detection surface, based on the difference in the 
center of gravity and the difference in the feature point; and 
15 outputting the control information corresponding to the rotation angle. 

This control information generation method may comprise calculating a 
rotation angle around a third axis which is perpendicular to the first axis and the second 
axis on the detection surface by using a rotation angle between a feature point of an 
image of the object to be detected in a frame at least one frame before the current frame 
20 and the extracted feature point around a reference point. 

This control information generation method may comprise outputting the 
control information corresponding to the movement in the first axis direction or the 
second axis direction by using the difference in the feature point and the rotation angle 
around the first axis or the second axis. 

25 
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